Traditional processor designs make use of various cache structures to store local copies of instructions and data in order to avoid lengthy access times of typical DRAM memory. In a typical cache hierarchy, caches closer to the processor (L1) tend to be smaller and very fast, while caches closer to the DRAM (L2 or L3) tend to be significantly larger but also slower (longer access time). The larger caches tend to handle both instructions and data, while quite often a processor system will include separate data cache and instruction cache at the L1 level (i.e. closest to the processor core). As is understood by those familiar with cache organization and operation, the processor core will issue requests which are handled, if possible, in the closest cache within which responsive data is found (a “cache hit”), with requests being “handed down” from one layer to the next as there may be cache misses.
All of these caches typically have similar organization, with the main difference being in specific dimensions (e.g. cache line size, number of ways per congruence class, number of congruence classes). In the case of an L1 Instruction cache, the cache is accessed either when code execution reaches the end of the previously fetched cache line or when a taken (or at least predicted taken) branch is encountered within the previously fetched cache line. In either case, a next instruction address is presented to the cache. In typical operation, a congruence class is selected via an abbreviated address (ignoring high-order bits), and a specific way within the congruence class is selected by matching the address to the contents of an address field within the tag of each way within the congruence class. Addresses used for indexing and for matching tags can use either effective or real addresses depending on system issues beyond the scope of this disclosure. Typically, low order address bits (e.g. selecting specific byte or word within a cache line) are ignored for both indexing into the tag array and for comparing tag contents. This is because for conventional caches, all such bytes/words will be stored in the same cache line.
Recently, Instruction Caches that store traces of instruction execution have been used, most notably with the Intel Pentium 4. These “Trace Caches” typically combine blocks of instructions from different address regions (i.e. that would have required multiple conventional cache lines). The objective of a trace cache is to handle branching more efficiently, at least when the branching is well predicted. The instruction at a branch target address is simply the next instruction in the trace line, allowing the processor to execute code with high branch density just as efficiently as it executes long blocks of code without branches. Just as parts of several conventional cache lines may make up a single trace line, several trace lines may contain parts of the same conventional cache line. Because of this, the tags must be handled differently in a trace cache. In a conventional cache, low-order address lines are ignored, but for a trace line, the full address must be used in the tag.
A related difference is in handling the index into the cache line. For conventional cache lines, the least significant bits are ignored in selecting a cache line (both index & tag compare), but in the case of a branch into a new cache line, those least significant bits are used to determine an offset from the beginning of the cache line for fetching the first instruction at the branch target. In contrast, the address of the branch target will be the first instruction in a trace line. Thus no offset is needed. Flow-through from the end of the previous cache line via sequential instruction execution simply uses an offset of zero since it will execute the first instruction in the next cache line (independent of whether it is a trace line or not). The full tag compare will select the appropriate line from the congruence class. In the case where the desired branch target address is within a trace line but not the first instruction in the trace line, the trace cache will declare a miss, and potentially construct a new trace line starting at that branch target.
One limitation of trace caches is that branch prediction must be reasonably accurate before constructing traces to be stored in a trace cache. For most code execution, this simply means delaying construction of traces until branch history has been recorded long enough to insure accurate prediction. However, some code paths contain branches that are inherently not predictable. For example, dynamic data may determine whether a specific branch is taken or not. Some implementations add a conventional L1 Instruction cache in addition to an L1 Instruction trace cache in order to better handle code that doesn't execute predictably. This is not a very efficient implementation since the trace cache must be sized to handle all desired trace lines and the conventional cache must be sized to handle all instructions that don't execute predictably. Since the balance between these two resources may be different for different applications, such a design will typically not achieve full utilization of both resources.